Performance improvement of dialog speech translation by rejecting unreliable utterances
نویسندگان
چکیده
We discuss how to measure the reliability of recognized utterances based on a confidence measure, and applied it to a dialog speech translation system. In this study, we employ generalized word posterior probability (GWPP), a confidence measure for verifying recognized words, and expand it to measure the reliability of recognized utterances. We confirmed the performance improvement by applying the rejection technique to a dialog speech translation system from Japanese to English. We conducted two kinds of performance evaluation. One is a ranking evaluation of translation output by human evaluators. The other is to measure the machine output against human results by a paired-comparison method. Both of them yield significant improvements.
منابع مشابه
A method for evaluating task-oriented spoken dialog translation systems based on communication efficiency
We propose a method for measuring communication efficiency from the viewpoint of conveying essential information in taskoriented spoken dialog translation. We present the results of one dialog experiment using speech-to-speech translation systems and a similar experiment using the Wizard of Oz method, which was carried out using hidden interpreters instead of a speech-to-speech translation syst...
متن کاملEfficient Language Model Construction for Spoken Dialog Systems by Inducting Language Resources of Different Languages
Since the quality of the language model directly affects the performance of the spoken dialog system (SDS), we should use a statistical language model (LM) trained with a large amount of data that is matched to the task domain. When porting an SDS to another language, however, it is costly to re-collect a large amount of user utterances in the target language. We thus use the language resources...
متن کاملLocalization of speech recognition in spoken dialog systems: how machine translation can make our lives easier
The localization of speech recognition for large-scale spoken dialog systems can be a tremendous exercise. Usually, all involved grammars have to be translated by a language expert, and new data has to be collected, transcribed, and annotated for statistical utterance classifiers resulting in a time-consuming and expensive undertaking. Often though, a vast number of transcribed and annotated ut...
متن کاملInteractive Error Resolution Strategies for Speech-to-Speech Translation Systems
1 In this demonstration, we will showcase BBN’s Speech-to-Speech (S2S) translation system that employs novel interaction strategies to resolve errors through user-friendly dialog with the speaker. The system performs a series of analysis on input utterances to detect out-ofvocabulary (OOV) named-entities and terms, sense ambiguities, homophones, idioms and ill-formed inputs. This analysis is us...
متن کاملChapter in Dialogue Processing in Spoken Language Systems
JANUS is a multilingual speech-to-speech translation system designed to facilitate communication between two parties engaged in a spontaneous conversation in a limited domain. In this paper we describe our methodology for evaluating translation performance. Our current focus is on end-to-end evaluations-the evaluation of the translation capabilities of the system as a whole. The main goal of ou...
متن کامل